EMBEDDING MODEL

RAG4ITOps: A Supervised Fine-Tunable and Comprehensive RAG Framework for IT Operations and Maintenance (2024.emnlp-industry)

Challenge:	Existing knowledge base embedding models are incomplete, i.e., missing a lot of valid triples.
Approach:	They propose a convolutional neural network embedding model for knowledge base completion that captures global relationships and transitional characteristics.
Outcome:	The proposed model outperforms state-of-the-art models on two benchmark datasets.

Copied to clipboard

Tianyang Zhang, Zhuoxuan Jiang, Shengguang Bai, Tianrui Zhang, Lin Lin, Yang Liu, Jiawei Ren

Natural Language Interfaces

Challenge:	Large Language Models (LLMs) have improved the open-domain QA’s performance, but how to efficiently handle enterprise-exclusive corpora and build domain-specific QA systems are still not studied for industrial applications.
Approach:	They propose a general and comprehensive framework based on Retrieval Augmented Generation (RAG) and facilitate the whole business process of establishing QA systems for IT operations and maintenance.
Outcome:	The proposed framework achieves superior results on two kinds of QA tasks.

Operational Advice for Dense and Sparse Retrievers: HNSW, Flat, or Inverted Indexes? (2025.acl-industry)

Copied to clipboard

Jimmy Lin

Information Retrieval

Challenge:	Currently, practitioners working on dense retrieval face a bewildering number of choices.
Approach:	They propose a framework for thinking about retrieval in terms of nearest-neighbor search over vector representations where these representations can be dense (typically called embeddings, generated from transformers) or flat (with brute-force search)
Outcome:	The proposed model explicates tradeoffs between HNSW and flat indexes from the perspectives of indexing time, query evaluation performance, and retrieval quality.

Leveraging Meta-Embeddings for Bilingual Lexicon Extraction from Specialized Comparable Corpora (C18-1)

Copied to clipboard

Amir Hazem, Emmanuel Morin

DRUM: Learning Demonstration Retriever for Large MUlti-modal Models (2025.acl-srw)

Challenge:	Recent studies on bilingual lexicon extraction from specialized comparable corpora show differences in performance . lack of large specialized corporan to build efficient representations can be partially explained .
Approach:	They propose to use character-based embedding models to combine different embeddable models . they emphasize how character-driven embeddance models outperform other models on quality .
Outcome:	The proposed model outperforms other models on quality of extracted bilingual lexicons . comparable corpora are an interesting and practical alternative to parallel corporation .

Copied to clipboard

Ellen Yi-Ge, Jiechao Gao, Wei Han, Wei Zhu

Examining Multilingual Embedding Models Cross-Lingually Through LLM-Generated Adversarial Examples (2025.findings-emnlp)

Challenge:	Recent studies have shown that large language models (LLMs) have impressive capabilities in dealing with new tasks with the help of in-context learning (ICL).
Approach:	They propose to concate the image and text embeddings to enhance the retrieval performance of a visual-language task and to calculate a list-wise ranking loss for training the embeddable model.
Outcome:	The proposed framework fine-tunes the CLIP embedding model to better meet the needs of the large vision-language models.

Copied to clipboard

Andrianos Michail, Simon Clematide, Rico Sennrich

Hierarchical Level-Wise News Article Clustering via Multilingual Matryoshka Embeddings (2025.acl-long)

Challenge:	Cross-Lingual Semantic Discrimination (CLSD) is a lightweight evaluation task that requires only parallel sentences and a Large Language Model (LLM) to generate adversarial distractors.
Approach:	They propose a lightweight task that requires only parallel sentences and a Large Language Model (LLM) to generate adversarial distractors.
Outcome:	The proposed task requires only parallel sentences and a Large Language Model (LLM) to generate adversarial distractors.

Copied to clipboard

Hans William Alexander Hanley, Zakir Durumeric

Information Extraction & Text Mining

Challenge:	Contextual large language model embeddings are often monolingual, do not scale, and struggle in multilingual settings.
Approach:	They propose a hierarchical approach to embed news articles and social media data using Matryoshka embeddings that can determine story similarity at varying levels of granularity based on which subset of dimensions is examined.
Outcome:	The proposed model achieves state-of-the-art performance on the SemEval 2022 task 8 dataset.

M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation (2024.findings-acl)

Copied to clipboard

Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, Zheng Liu

Generalized Embedding Models for Industry 4.0 Applications (2025.emnlp-industry)

Challenge:	In this paper, we introduce a new embedding model for semantic retrieval of more than 100 working languages.
Approach:	They propose a new embedding model that supports multi-lingual, cross-lingual and long-document retrieval . they propose integrating relevance scores from different retrieval functionalities into the teacher signal .
Outcome:	The proposed model exhibits superior performance on multilingual, cross-lingual, and long-document retrieval benchmarks.

Copied to clipboard

Christodoulos Constantinides, Shuxin Lin, Dhaval C Patel

ICLER: Intent CLassification with Enhanced Reasoning (2025.findings-emnlp)

Challenge:	Using Large Language Models (LLMs) to automate tasks has emerged as the next frontier of innovation.
Approach:	They propose a model that generalizes to queries involving similar assets and retrieves relevant items from natural language tasks.
Outcome:	The proposed model can be used to generalize to queries involving similar assets, such as identifying sensors relevant to an asset’s failure mode.

Copied to clipboard

Dezheng Gao, Dong Xiaozheng, SHuangtao Yang, Bo Fu

Information Retrieval

Challenge:	Existing methods for intent classification are inadequate in identifying micro-grained intentions . ICLER is based on In-Context Learning, but it is inadequate in enterprise vertical domains .
Approach:	They propose an intent classification method with enhanced reasoning that optimizes the embedding model to capture subtle sentence-level information.
Outcome:	The proposed method outperforms existing methods in intent identification tasks in vertical domains.

Understanding Undesirable Word Embedding Associations (P19-1)

Copied to clipboard

Kawin Ethayarajh, David Duvenaud, Graeme Hirst

Compressing then Matching: An Efficient Pre-training Paradigm for Multimodal Embedding (2026.acl-long)

Challenge:	Word embeddings are often criticized for capturing undesirable word associations such as gender stereotypes.
Approach:	They propose to use subspace projection to debias vectors post hoc using a model that implicitly does matrix factorization to debunk gender bias.
Outcome:	The proposed test overestimates gender bias in word embeddings by using subspace projection, a method that is widely used in training.

Copied to clipboard

Da Li, Yuxiao Luo, Keping Bi, Jiafeng Guo, Wei Yuan, Biao Yang, Yan Wang, Fan Yang, Tingting Gao, Guorui Zhou

VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval (2024.acl-long)

Challenge:	Recent approaches demonstrate that MLLMs can be adapted into competitive embedding models via large-scale contrastive learning.
Approach:	They propose a compressed pre-training phase which serves as a warm-up stage for contrastive learning.
Outcome:	The proposed model achieves state-of-the-art among MLLMs of comparable size on the MMEB, realizing optimization in both efficiency and effectiveness.

Copied to clipboard

Junjie Zhou, Zheng Liu, Shitao Xiao, Bo Zhao, Yongping Xiong

A Capsule Network-based Embedding Model for Knowledge Graph Completion and Search Personalization (N19-1)

Challenge:	Existing methods for multimodal retrieval are mostly text-oriented, which lack the capability to process visual information.
Approach:	They propose a multi-modal multi-text embedding model VISTA which extends a powerful text encoder with the image understanding capability by introducing visual token embedds.
Outcome:	The proposed model achieves superior performance across a variety of multi-modal retrieval tasks in zero-shot and supervised settings.

Copied to clipboard

Dai Quoc Nguyen, Thanh Vu, Tu Dinh Nguyen, Dat Quoc Nguyen, Dinh Phung

Transferable Embedding Inversion Attack: Uncovering Privacy Risks in Text Embeddings without Model Queries (2024.acl-long)

Challenge:	Existing knowledge graphs with billions of triples are incomplete, i.e., missing a lot of valid triples.
Approach:	They propose to embed relationship triples into a capsule network using a convolution layer and multiple filters to generate feature maps.
Outcome:	The proposed model outperforms strong search personalization baselines on two benchmark datasets and outperformed previous state-of-the-art models on WN18RR and FB15k-237.

Copied to clipboard

Yu-Hsiang Huang, Yuche Tsai, Hsiang Hsiao, Hong-Yi Lin, Shou-De Lin

Co-Evolving LLMs and Embedding Models via Density-Guided Preference Optimization for Text Clustering (2025.emnlp-main)

Challenge:	Recent advances in text embedding models have significantly streamlined the process of generating embeddables.
Approach:	They develop a transfer attack method that uses a surrogate model to mimic the victim model's behavior and infers sensitive information from embeddings without direct access.
Outcome:	The proposed method outperforms existing methods and reveals potential privacy vulnerabilities in embedding technologies.

Copied to clipboard

Zetong Li, Qinliang Su, Minhua Huang, Yin Yang

GASE: Generatively Augmented Sentence Encoding (2025.findings-emnlp)

Challenge:	Existing methods for text clustering use static pseudo-oracles, i.e., unidirectionally querying them for similarity assessment or data augmentation.
Approach:	They propose a training framework that enables bidirectional refinement between LLMs and embedding models by using task-aware prompts to guide the LLM in generating interpretations for the input texts.
Outcome:	Experiments on 14 benchmark datasets across 5 tasks demonstrate the effectiveness of the proposed training framework.

Copied to clipboard

Manuel Frank, Haithem Afli

A Relational Memory-based Embedding Model for Triple Classification and Search Personalization (2020.acl-main)

Challenge:	Generatively Augmented Sentence Encoding variates the input text by paraphrasing, summarizing, or extracting keywords, followed by pooling the original and synthetic embeddings.
Approach:	They propose a training-free approach to improve sentence embeddings by applying generative text models for data augmentation at inference time.
Outcome:	The proposed approach does not require access to model parameters or computational resources typically required for fine-tuning state-of-the-art models.

Copied to clipboard

Dai Quoc Nguyen, Tu Nguyen, Dinh Phung

Information Retrieval

Challenge:	Existing knowledge graph embedding methods do not allow for the prediction of new triples, such as for search personalization tasks.
Approach:	They propose a relational memory network to encode potential dependencies in relationship triples by a transformer self-attention mechanism.
Outcome:	The proposed model obtains state-of-the-art results on SEARCH17, WN11 and FB13 for the search personalization task, and on a convolutional neural network-based decoder.

Word and Document Embedding with vMF-Mixture Priors on Context Word Vectors (P19-1)

Copied to clipboard

Shoaib Jameel, Steven Schockaert

Embedding Imputation with Grounded Language Information (P19-1)

Challenge:	Word embedding models typically learn two types of vectors: target word vectors and context word vector.
Approach:	They propose to explicitly impose a cluster structure on context word vectors to improve word embedding models.
Outcome:	The proposed model improves word embedding models qualitatively by imposing a cluster structure on the set of context word vectors.

Copied to clipboard

Ziyi Yang, Chenguang Zhu, Vin Sachidananda, Eric Darve

Claim Matching Beyond English to Scale Global Fact-Checking (2021.acl-long)

Challenge:	Existing approaches to embedding imputation use vector space properties or subword information to learn representations for rare or unseen words.
Approach:	They propose an online method to construct a knowledge graph from grounded information and an algorithm to map from the resulting graph to the space of the pre-trained embeddings.
Outcome:	The proposed method improves on a card-660 task by 11% and 17.8% respectively using GloVe embeddings.

Copied to clipboard

Ashkan Kazemi, Kiran Garimella, Devin Gaffney, Scott Hale

Improving Hate Speech Detection with Deep Learning Ensembles (L18-1)

Challenge:	Existing methods to fact-check content are not scaled well in non-English contexts.
Approach:	They propose to use a WhatsApp tipline and public group message dataset to find pairs of textual messages containing claims that can be served with one fact-check.
Outcome:	The proposed model outperforms existing models in English, Hindi, and Tamil in all settings.

Copied to clipboard

Steven Zimmerman, Udo Kruschwitz, Chris Fox

Cross-lingual Feature Extraction from Monolingual Corpora for Low-resource Unsupervised Bilingual Lexicon Induction (2022.coling-1)

Challenge:	censorship is a potential risk when addressing these issues with automated text classification methods.
Approach:	They propose to use a neural network-based ensemble method to better classify hate speech using a publicly available embedding model and a popular sentiment dataset.
Outcome:	The proposed method improves by 5 points on a hate speech corpus from Twitter and a popular sentiment dataset.

Copied to clipboard

Zihao Feng, Hailong Cao, Tiejun Zhao, Weixuan Wang, Wei Peng

Multilinguality

Challenge:	Unsupervised bilingual lexicon induction models fail on low-resource language pairs due to insufficient initialization.
Approach:	They propose a method to learn cross-lingual features from monolingual corpora for low-resource UBLI by integrating cross-linguistic representations with pre-trained word embeddings in a fully unsupervised initialization.
Outcome:	The proposed method outperforms state-of-the-art methods on low-resource language pairs and improves representational ability and robustness of existing embedding models.

Structure Aware Negative Sampling in Knowledge Graphs (2020.emnlp-main)

Copied to clipboard

Kian Ahrabian, Aarash Feizi, Yasmin Salehi, William L. Hamilton, Avishek Joey Bose

Multimodality

Challenge:	Existing methods for learning low-dimensional representations of entities and relations in knowledge graphs employing corruption distributions that generate hard negative samples.
Approach:	They propose a structure-aware negative sampling strategy that utilizes the rich graph structure by selecting negative samples from a node’s k-hop neighborhood.
Outcome:	The proposed method finds semantically meaningful negatives and is competitive with SOTA approaches while requires no additional parameters nor difficult adversarial optimization.

Efficient Integration of External Knowledge to LLM-based World Models via Retrieval-Augmented Generation and Reinforcement Learning (2025.findings-emnlp)

Copied to clipboard

Chang Yang, Xinrun Wang, Qinggang Zhang, Qi Jiang, Xiao Huang

Learning Gender-Neutral Word Embeddings (D18-1)

Challenge:	Existing attempts to enhance LLM-based world models through prompting or fine-tuning approaches are either requiring human knowledge or computationally extensive.
Approach:	They propose a framework that leverages retrieval-augmented generation to integrate external knowledge to LLM-based world models.
Outcome:	The proposed framework outperforms baseline models and exhibits strong generalizability.

Copied to clipboard

Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, Kai-Wei Chang

Enhancing Deep Learning with Embedded Features for Arabic Named Entity Recognition (2022.lrec-1)

Challenge:	Word embeddings trained on human-generated corpora inherit strong gender stereotypes . prior studies show such embeddables exhibit social biases, such as gender stereotype .
Approach:	They propose a method to preserve gender information in certain dimensions of word vectors . they propose GN-GloVe, which is a gender-neutral variant of the word embedding model .
Outcome:	The proposed method preserves gender information in certain dimensions of word vectors while compelling other dimensions to be free of gender influence.

Copied to clipboard

Ali L. Hatab, Caroline Sabty, Slim Abdennadher

Retrieval Augmented Fact Verification by Synthesizing Contrastive Arguments (2024.acl-long)

Challenge:	Word embeddings can capture the semantics of words and other hidden features, but the Arabic language is complex and requires a large amount of information to process.
Approach:	They propose to add morphological and syntactical features to Arabic word embeddings to train the model.
Outcome:	The proposed model outperforms the previous systems to the best of our knowledge.

Copied to clipboard

Zhenrui Yue, Huimin Zeng, Lanyu Shang, Yifan Liu, Yang Zhang, Dong Wang

Bridging the Language Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs (2025.coling-main)

Challenge:	Existing methods to verify claim credibility rely on embedded knowledge or unreliable context.
Approach:	They propose retrieval augmented fact verification through the synthesis of contrasting arguments (RAFTS) they use an embedding model to identify informative demonstrations and in-context prompts to generate the prediction and explanation.
Outcome:	The proposed method outperforms existing methods with smaller LLMs or unreliable contexts.

Copied to clipboard

Somnath Kumar, Vaibhav Balloli, Mercy Ranjit, Kabir Ahuja, Sunayana Sitaram, Kalika Bali, Tanuja Ganu, Akshay Nambi

An LLM-based Framework for Biomedical Terminology Normalization in Social Media via Multi-Agent Collaboration (2025.coling-main)

Challenge:	Large language models (LLMs) excel in diverse applications but still struggle with non-Latin scripts and low-resource languages.
Approach:	They propose a dynamic learning approach that optimizes prompt strategy, embedding model, and LLM per query at runtime.
Outcome:	The proposed approach achieves 10-15% improvements in multilingual performance over pre-trained models and 4x gains compared to fine-tuned, language-specific models.

Copied to clipboard

Yongqi Fan, Kui Xue, Zelin Li, Xiaofan Zhang, Tong Ruan

LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning (2025.findings-emnlp)

Challenge:	Experimental results indicate that our approach exhibits competitive performance.
Approach:	They propose a tuning-free approach to normalize non-standard terms using large language models . they use a search engine and a domain knowledge base to expand the short texts into accurate descriptions .
Outcome:	The proposed approach is based on the "Recall and Re-rank" framework . it can be used to identify the standard term in a specified termbase for non-standardized mentions .

Copied to clipboard

Zhibin Lan, Liqiang Niu, Fandong Meng, Jie Zhou, Jinsong Su

Skill-Based Few-Shot Selection for In-Context Learning (2023.emnlp-main)

Challenge:	Existing LMM-based embedding models exhibit a high degree of overlap in similarity distribution between positive and negative pairs, making it challenging to distinguish hard negative pairs effectively.
Approach:	They propose a framework that improves the embedding model's representation learning for negative pairs based on their discriminative difficulty.
Outcome:	The proposed framework improves the embedding model's representation learning for negative pairs based on their discriminative difficulty.

Copied to clipboard

Shengnan An, Bo Zhou, Zeqi Lin, Qiang Fu, Bei Chen, Nanning Zheng, Weizhu Chen, Jian-Guang Lou

A Comparison of Independent and Joint Fine-tuning Strategies for Retrieval-Augmented Generation (2025.findings-emnlp)

Challenge:	Existing methods based on pre-trained embeddings can be easily biased by surface features that are not important for the target task.
Approach:	They propose a skill-based few-shot selection method for in-context learning . it generates skill-specific descriptions for each test case and candidate example .
Outcome:	The proposed method significantly outperforms existing methods in five cross-domain semantic parsing datasets and six backbone models.

Copied to clipboard

Neal Gregory Lawton, Alfy Samuel, Anoop Kumar, Daben Liu

KARPA: A Training-free Method of Adapting Knowledge Graph as References for Large Language Model’s Reasoning Path Aggregation (2025.findings-acl)

Challenge:	Multiple fine-tuning strategies exist with different costs and benefits for RAG pipelines.
Approach:	They evaluate several RAG fine-tuning strategies with different costs and benefits . embedding and generator models can be fine- tuned to increase performance .
Outcome:	The proposed techniques improve quality metrics, but have different computational costs.

Copied to clipboard

Siyuan Fang, Kaijing Ma, Tianyu Zheng, Xeron Du, Ningxuan Lu, Ge Zhang, Qingkun Tang

Negative Matters: Multi-Granularity Hard-Negative Synthesis and Anchor-Token-Aware Pooling for Enhanced Text Embeddings (2025.acl-long)

Challenge:	Existing methods for large language models (LLMs) are limited by step-by-step decision-making on KGs, or require fine-tuning or pre-training on specific KG.
Approach:	They propose a framework that harnesses the global planning abilities of large language models (LLMs) for efficient and accurate KG reasoning.
Outcome:	Extensive experiments show that the proposed framework achieves state-of-the-art performance in KGQA tasks, delivering both high efficiency and accuracy.

Copied to clipboard

Tengyu Pan, Zhichao Duan, Zhenyu Li, Bowen Dong, Ning Liu, Xiuxing Li, Jianyong Wang

RATE: Reviewer Profiling and Annotation-free Training for Expertise Ranking in Peer Review Systems (2026.acl-long)

Challenge:	Text embedding models are used for various natural language processing tasks such as sentiment analysis, text clustering, and content-based information retrieval.
Approach:	They propose a synthesis framework that leverages large language models to generate diverse negative samples with varying levels of similarity with the query.
Outcome:	The proposed framework achieves state-of-the-art performance surpassing existing synthesis strategies with synthetic data and when combined with public retrieval datasets.

Copied to clipboard

Weicong Liu, Zixuan Yang, Yibo Zhao, Xiang Li